Image completion or inpainting (hereinafter “image completion” for brevity) is a popular image editing tool for object removal and replacement or digital photograph restoration. In most approaches, image completion is used to fill holes after unwanted objects are removed. Image completion also may be used to extend an image beyond its original boundaries.
For example, casually shot panoramas often have irregular boundaries. Most users, however, prefer images with rectangular boundaries. One trivial solution implemented by conventional stitching software is to crop the image to the largest box that is fully contained within the panorama, however, this simple solution often removes large parts of the image. An alternative is to apply any existing completion algorithm to fill the missing regions of the panorama bounding box.
However, existing image completion algorithms often fail. One type of failure typically appears as an inability to synthesize some textures well, e.g., noticeable as blurring and/or mismatching in the image. Another type of failure is when image completion provides results that are semantically implausible; (e.g., the completed sky in a scenic image appears with some non-sky scenery, such as the ground, floating in the air above the horizon). Any image completion technology that avoids such failures to a greater extent is desirable.